[SPARK-59435][UDF] Qualify UDF worker proto import paths - #58738
Closed
haiyangsun-db wants to merge 2 commits into
Closed
haiyangsun-db wants to merge 2 commits into
haiyangsun-db wants to merge 2 commits into
Conversation
HyukjinKwon
approved these changes
Sep 14, 2026
HyukjinKwon
pushed a commit
that referenced
this pull request
Sep 14, 2026
### What changes were proposed in this pull request? This PR relocates the UDF worker protobuf definitions beneath `udf/worker/proto/src/main/protobuf/org/apache/spark/udf/worker/` and updates their intra-proto imports to use the corresponding qualified paths. This follows the namespaced source layouts used by other Spark protobuf definitions, including: - [Spark Connect common.proto](https://github.com/apache/spark/blob/master/sql/connect/common/src/main/protobuf/spark/connect/common.proto) - [configuration schema](https://github.com/apache/spark/blob/master/common/config/src/main/protobuf/org/apache/spark/config/config_schema.proto) - [status store types](https://github.com/apache/spark/blob/master/core/src/main/protobuf/org/apache/spark/status/protobuf/store_types.proto) - [streaming state message](https://github.com/apache/spark/blob/master/sql/core/src/main/protobuf/org/apache/spark/sql/execution/streaming/StateMessage.proto) ### Why are the changes needed? The UDF worker definitions previously used generic bare import names such as `common.proto` and `udf_message.proto`. Relocating them establishes stable, qualified import paths, avoiding ambiguity if Spark protobuf sources are later collected, indexed, or compiled alongside definitions with similarly named files. The message definitions, generated Java package, and wire format are unchanged. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? - `protoc` compilation of the relocated UDF worker protobuf definitions. - `build/mvn -pl udf/worker/proto,udf/worker/grpc -am -DskipTests compile` - `build/mvn -pl udf/worker/grpc -am test` - UDF worker core: 59 tests passed. - UDF worker gRPC: 60 tests passed. ### Was this patch authored or co-authored using generative AI tooling? Yes. Closes #58738 from haiyangsun-db/SPARK-59435-qualify-udf-worker-proto-imports. Authored-by: Haiyang Sun <haiyang.sun@databricks.com> Signed-off-by: Hyukjin Kwon <hyukjin.kwon@databricks.com> (cherry picked from commit f3b1883) Signed-off-by: Hyukjin Kwon <hyukjin.kwon@databricks.com>
Member
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
This PR relocates the UDF worker protobuf definitions beneath
udf/worker/proto/src/main/protobuf/org/apache/spark/udf/worker/and updatestheir intra-proto imports to use the corresponding qualified paths.
This follows the namespaced source layouts used by other Spark protobuf
definitions, including:
Why are the changes needed?
The UDF worker definitions previously used generic bare import names such as
common.protoandudf_message.proto. Relocating them establishes stable,qualified import paths, avoiding ambiguity if Spark protobuf sources are later
collected, indexed, or compiled alongside definitions with similarly named
files.
The message definitions, generated Java package, and wire format are unchanged.
Does this PR introduce any user-facing change?
No.
How was this patch tested?
protoccompilation of the relocated UDF worker protobuf definitions.build/mvn -pl udf/worker/proto,udf/worker/grpc -am -DskipTests compilebuild/mvn -pl udf/worker/grpc -am testWas this patch authored or co-authored using generative AI tooling?
Yes.